\section{Performance Testing}
\subsection{Experimental Setup}
Performance Testing was performed on the application deployed on the Google App Engine infrastructure at the URL \url{http://lak-shay.appspot.com}. We utilized the two sets of workloads provided for performance testing - read skewed (with close to 75\% read operations) and mixed (with a 50-50 mixture of read write operations). We ran the first 1000 operations from each file. In order to accommodate for transactions which do no have commit operations within the first 1000 operations, we ignore transactions which do not finish. In the initial performance testing of SS2PL loads, we found that for a number of loads very few transactions were able to complete, in some cases only one. This was because of a deadlock situation during the load. In order to resolve this issue we implemented a function to abort transactions which have a transaction waiting for longer than 20 seconds. An HTTP request to call this function was inserted per 100 operations in the SS2PL Loads. 
The following measurements were taken for each of the workloads:
\begin{itemize}
\item \textbf{SS2PL} 
	\begin{itemize}
		\item Transaction Complete Times - Minimum, Maximum and Average
		\item Operation wait times - Minimum, Maximum and Average wait times. The amount of time an operation has to wait in the wait queue before it can obtain the lock. This is only relevant for operations that cannot obtain the lock when they are submitted to the system. This measurement gives a pointer to the contention in the system.
	\end{itemize}
\item \textbf{OCC}
	\begin{itemize}
	\item Transaction Complete Times - Minimum, Maximum and Average
	\item Transaction Aborts - Since there is no concept of waiting in case of OCC, the measure of contention for OCC is the no. of aborted transactions.
	\end{itemize}
\end{itemize}
\subsection{Testing Difficulties} 
We encountered the following difficulties during our testing:
\begin{itemize}
	\item \textbf{Data Store Timeouts} - The performance of the data store gradually deteriorates as the no. of entities increase. This results in timeouts in the middle of iterating over the results from the query. This complicates matters as the recovery from these errors has to take into account, the current state before attempting a retry. In the absence of recovery, data time outs result in an inconsistent state of the data store which might affect acquiring locks in case of SS2PL especially if a commit operation of a transaction holding a lock encounters an error. 
	\item \textbf{OCC Parallel Validation} - Since the httpload was executed with the \textit{-parallel 1} option, it resulted in operations being submitted one by one to the application. This prevents testing the Parallel validation of OCC as no two commit operations enter the system simultaneously and hence no two transactions are in the val-write phase at the same time
	\item \textbf{Network Latency Issues} -  Since the time measurements are dependent on the rate at which operations are submitted, the network latency at the time of performing the tests also has an affect on these measurements.
\end{itemize}
\subsection{Results}
The results of performance testing are detailed in the tables~\ref{table:ss2pl} and ~\ref{table:occ}. There are no wait time calculation for OCC Transaction as an OCC operations never waits for execution.
\begin{table*}
\caption{SS2PL Performance Testing Results}
\label{table:ss2pl}
\begin{tabular}{|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|}
\hline
\textbf{Title} &	\textbf{Throughput (fetches/s)} &		\textbf{Min Transaction Complete Time(s)} &	\textbf{Max Transaction Complete Time(s)} &	\textbf{Avg Transaction Complete Time(s)} &	\textbf{Min Operation Wait Time(s)} &	\textbf{Max Operation Wait Time(s)} &	\textbf{Avg Operation Wait Time(s)} & \textbf{Aborted (\%)} & \textbf{Total Completed Transactions}\\
\hline Read Skewed-1 &  1.21 & 11.71 & 237.22 & 76.50 & 7.56 & 195.97 & 53.23 & 42.22\% & 90\\
\hline Read Skewed-2 &  1.40 & 13.64 & 135.74 & 44.44 & 2.17 & 51.05 & 24.90 & 57.46\% & 94\\
\hline Read Skewed-3 &  1.42 & 11.68 & 132.57 & 50.98 & 2.01 & 120.58 & 49.93 & 41.50\% & 106\\
\hline Read Skewed-4  & 1.49 & 14.56 & 178.67 & 45.04 & 2.66 & 90.50 & 32.430 & 38.77\% & 98\\
\hline Read Skewed-5  & 1.54 & 10.76 & 100.06 & 38.63 & 2.98 & 101.10 & 36.190 & 34.26\%& 108\\
\hline Mixed-1  & 1.73 & 15.74 & 370.76 & 217.71 & 70.06 & 90.25 & 46.64 & 32.5\%& 40\\
\hline Mixed-2  & 1.60 & 23.58 & 371.17 & 213.77 & 41.42 & 88.43 & 33.63 & 27.08\%& 48\\
\hline Mixed-3  & 1.59 & 15.19 & 363.93 & 202.88 & 20.36 & 231.35 & 73.01 & 27.66& 47\\
\hline Mixed-4  & 1.56 & 21.04 & 358.50 & 217.03 & 94.14 & 146.07 & 54.51 & 43.47\%& 46\\
\hline Mixed-5 & 1.57 & 78.51 & 360.32 & 239.74 & 11.74 & 87.63 & 37.07 & 49.53\%& 43\\
\hline
\end{tabular}
\end{table*}

\begin{table*}
\caption{OCC Performance Testing Results}
\label{table:occ}
\begin{tabular}{|p{2cm}|p{2cm}|p{2cm}|p{2cm}|p{2cm}|p{2cm}|p{2cm}|}
\hline
\textbf{Load} & \textbf{Throughput (fetches/s)} & \textbf{Min Transaction Complete Time(s)}	& \textbf{Max Transaction Complete Time(s)} & \textbf{Avg Transaction Complete Time(s)} & \textbf{Aborted Transactions (\%)} & \textbf{Total Completed Transactions}\\
\hline Read Skewed-1 & 2.70 & 7.16 & 34.27 & 17.12 & 37.03\% & 108\\
\hline Read Skewed-2 & 1.86 & 11.54 & 54.99 & 23.40 & 53.27\% & 107\\
\hline Read Skewed-3 & 1.83 & 11.17 & 49.61 & 23.80 & 44.44\% & 108\\
\hline Read Skewed-4 & 1.88 & 13.36 & 49.56 & 25.33 & 44.44\% & 109\\
\hline Read Skewed-5 & 1.79 & 7.78 & 57.74 & 26.47 & 29.63\%  & 108\\
\hline Mixed-1 & 2.78 & 6.21 & 28.27 & 14.86 & 63.30\% & 109\\
\hline Mixed-2 & 1.83 & 7.97 & 39.03 & 21.48 & 60.18\% & 108\\
\hline Mixed-3 & 1.94 & 11.11 & 44.86 & 21.18 & 57.40\% & 108\\
\hline Mixed-4 & 1.56 & 10.12 & 52.763 & 25.24 & 58.88\% & 108\\
\hline Mixed-5 & 1.91 & 8.78 & 40.836 & 20.91 & 57.40\% & 108\\
\hline
\end{tabular}
\end{table*}

\subsection{Discussion}
The results highlight the tradeoff between OCC and SS2PL. The average transaction completion times for OCC, ranging from 17-26 seconds are much lower as compared to SS2PL where they range from 76-239 seconds. A part of the difference in the transaction completion times in OCC is due to the execution of the extra \textit{breakDeadlock} operation after every 100 operations. But despite that, the variation in the average completion times clearly suggests that OCC is better suited in an environment that requires strict time bounds on transaction execution. This might be especially true for applications tied to a user interface. The downside of OCC is the relatively high number of transaction aborts. The transaction aborts are especially noticeable in the mixed workloads for OCC where they range from 57-63\%. SS2PL performs much better in terms of transaction aborts in a mixed load situation. Although the penalty in case of SS2PL is paid again in terms of transaction completion times which are significantly higher than average completion times for read skewed workloads. This is obviously because of the waiting of the operations. Another interesting facet of SS2PL results in case of mixed workloads is that the number of transactions that actually complete by the time the 1000 operations are executed are in the range of 40-48 which is almost half the number of transactions executed in case of OCC. This further reinforces the observation that SS2PL with wait queue is not suited for environments which have hard or soft hard bounds on transaction completion times. But SS2PL serves well in situations where the actual completion is more important than the time taken for the completion of a transaction.

Another thing to note about SS2PL is that in case of mixed workloads, without any deadlock breaking strategy we could not get more than one transaction to complete. This highlights the importance of having deadlock resolution strategy for SS2PL.
